Papers with model generalization ability
Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing automatic question generation methods focus on encoding passage and answer to generate question. |
| Approach: | They propose an automatic question generation approach which integrates question generation with its dual problem, question answering, into a unified primal-dual framework. |
| Outcome: | The proposed approach outperforms existing methods on SQuAD and HotpotQA benchmarks. |
Unveiling Internal Reasoning Modes in LLMs: A Deep Dive into Latent Reasoning vs. Factual Shortcuts with Attribute Rate Ratio (2025.emnlp-main)
Copied to clipboard
Yiran Yang, Haifeng Sun, Jingyu Wang, Qi Qi, Zirui Zhuang, Huazheng Wang, Pengfei Ren, Jing Wang, Jianxin Liao
| Challenge: | Existing research in multi-hop questions has identified two reasoning modes, but has not investigated how these modes differ during inference. |
| Approach: | They propose a classification metric that compares latent reasoning and factual shortcuts in multi-hop questions. |
| Outcome: | The proposed metric achieves 90% accuracy on the proposed datasets and demonstrates effectiveness in RAG conflict scenarios. |
Improving Model Generalization: A Chinese Named Entity Recognition Case Study (2021.acl-short)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a fundamental building block for various downstream natural language processing tasks due to the ambiguous word boundaries and complex composition. |
| Approach: | They propose to resample entities within the same category to encourage a model to leverage both name and context knowledge in the training process. |
| Outcome: | The proposed method significantly improves a model’s ability to detect unseen entities, especially for company, organization and position categories. |
CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation methods for text summarization systems are limited to in-domain setting, where supervised pre-trained models are evaluated on the same dataset. |
| Approach: | They propose to use a cross-dataset evaluation approach to evaluate different summarization systems in a multi-domain setting. |
| Outcome: | The proposed model can be used to evaluate text summarization systems on different datasets. |
Reconstructing Capsule Networks for Zero-shot Intent Classification (D19-1)
Copied to clipboard
| Challenge: | Existing methods for intent classification are limited due to fast-emerging intents . a recent study shows that existing methods are not effective in recognizing unseen intents. |
| Approach: | They propose to reconstruct capsule networks for zero-shot intent classification by using latent information from labeled utterances. |
| Outcome: | The proposed method outperforms existing methods on two task-oriented dialogue datasets in different languages. |
A Rigorous Study on Named Entity Recognition: Can Fine-tuning Pretrained Model Lead to the Promised Land? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental task of information extraction. |
| Approach: | They propose to perform randomization tests on standard NER benchmarks to examine name regularity, mention coverage and context diversity. |
| Outcome: | The proposed model performs better on standard NER benchmarks than other models on open datasets. |
Unsupervised Non-transferable Text Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train a good deep learning model require labeled data for the target domain which can be difficult to obtain. |
| Approach: | They propose an unsupervised non-transferable learning method that does not require annotated target domain data and introduce a secret key component for recovering the model’s access to the target domain. |
| Outcome: | The proposed method reduces model generalization ability in specific target domains while still recovering access to the target domain. |
ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videos (2023.emnlp-main)
Copied to clipboard
Te-Lin Wu, Zi-Yi Dou, Qingyuan Hu, Yu Hou, Nischal Chandra, Marjorie Freedman, Ralph Weischedel, Nanyun Peng
| Challenge: | despite its importance, there are few datasets that cover multimodal counterfactual reasoning . a dataset focusing on this area is limited because of its limited coverage over synthetic environments . |
| Approach: | They develop a video question answering dataset that provides questions on multimodal reasoning . they ask questions about counterfactual hypotheses over visual events . |
| Outcome: | The proposed dataset shows a significant performance gap between models and humans . it provides questions that span physical, social, and temporal dimensions . |